User-Centred Requirements Handbook |
Wizard of Oz is a technique used to present advanced concepts of interactions to users. In essence an expert (the wizard), possibly located behind a screen, processes input from a user and emulates system output. The aim is to demonstrate computer capabilities which cannot be done by the computer, for technical reasons or lack of resources. For example, the expert may be someone behind the screen typing in responses to speech inputs from the user and thus pretending that the machine can understand this speech - just like the 'Wizard' in the eponymous film.
It is highly applicable to "intelligent interfaces" which feature agents, advisors and/or natural language processing.
• This method allows user requirements and usability issues to be explored at an early stage in the design process, particularly for systems which go beyond readily available technology.
• The member of the design team who plays the Wizard can gain valuable insights from the close involvement in the user's activity.
• The person playing the role of the Wizard must appreciate the functionality of the proposed system in order to provide a convincing representation.
• This method requires a higher commitment of resources than other approaches to prototyping such as those that rely on simple paper-based materials.
Two computer systems would be required, one each for the user and the wizard. Two staff are required to conduct the evaluation - one to play the wizard, another to instruct the user and record the session. The wizard should be an experienced member of the design team so that system responses are logical and not beyond the realms of possibility. The time overhead largely depends upon the task domain and the number of users exposed to the prototype.
The general procedure for implementing this method is outlined in the following.
1. Firstly, allow enough time to fabricate the Wizard of Oz prototype, design some tasks, recruit users, conduct the evaluation of the prototype and report the results.
2. Allocate the role of Wizard and the role of facilitator to the relevant staff.
3. Assemble the necessary equipment.
4. Develop the prototype itself.
5. Select appropriate users to test the prototype, trying to cover the range of users within the target population.
6. Prepare realistic task scenarios for the evaluation.
7. Pilot the evaluation procedure and ensure the Wizard is well practised in playing the role of the computer.
8. Ensure recording facilities are available and functioning.
9. Conduct each session. The facilitator instructs the user to work through the allocated tasks interacting and responding to the system as appropriate.
10. Conduct post-session interviews with the users, drawing upon questions and issues raised during the use of the prototype.
11. Debrief and thank the user.
12. Analyse information obtained, summarise observations and user evaluations. Consider the themes and severity of the problems identified.
13. Summarise design implications and recommendations for improvements and feed back to design team. Video recordings can support this.
14. Where necessary refine the prototype and repeat the above process.
• Explain to the users beforehand the Wizard of Oz idea and that the system is being operated by another person. This will avoid users later feeling they are being tricked by the evaluators.
• Practice beforehand to make sure that the Wizard can keep up with the user. If necessary slow the user down.
• Keep the session short to avoid tiring the Wizard too much.
Maulsby, Greenberg and Mander (1993), Nielsen (1993).